Combining Multi-modal Features for Social Media Analysis

In this chapter we discuss methods for efficiently modeling the diverse information carried by social media. The problem is viewed as a multi-modal analysis process where specialized techniques are used to overcome the obstacles arising from the heterogeneity of data. Focusing at the optimal combination of low-level features (i.e., early fusion), we present a bio-inspired algorithm for feature selection that weights the features based on their appropriateness to represent a resource. Under the same objective of optimal feature combination we also examine the use of pLSA-based aspect models, as the means to define a latent semantic space where heterogeneous types of information can be effectively combined. Tagged images taken from social sites have been used in the characteristic scenarios of image clustering and retrieval, to demonstrate the benefits of multi-modal analysis in social media. Spiros Nikolopoulos Informatics & Telematics Institute, Thermi, Thessaloniki, Greece and School of Electronic Engineering and Computer Science, Queen Mary University of London, E1 4NS, London, UK, e-mail: [email protected] Eirini Giannakidou Informatics & Telematics Institute, Thermi, Thessaloniki, Greece and Department of Computer Science, Aristotle University of Thessaloniki, Greece, e-mail: [email protected] Ioannis Kompatsiaris Informatics & Telematics Institute, Thermi, Thessaloniki, Greece, e-mail: [email protected] Ioannis Patras School of Electronic Engineering and Computer Science, Queen Mary University of London, E1 4NS, London, UK Tel. +44 20 7882 7523, Fax: +44 20 7882 7997, e-mail: [email protected] Athena Vakali Department of Computer Science, Aristotle University of Thessaloniki, Greece e-mail: [email protected]